Field of the Invention
The present invention relates to an image-processing apparatus and an image-processing method.
Description of the Related Art
Techniques exist for generating an image with a high dynamic range by combining a plurality of images including an image (a first frame) obtained by imaging with a relatively long exposure time and an image (a second frame) obtained by imaging with a relatively short exposure time. Such a technique (combination) is referred to as “high dynamic range combination (HDR combination)” and an image generated by HDR combination is referred to as a “high dynamic range combine image (HDR combine image)”. Performing HDR combination enables an HDR combine image to be generated in which occurrences of overshooting of data related to received light due to exposure time are suppressed. In other words, an HDR combine image in which occurrences of blown-out highlights and black defects are suppressed can be generated.
In recent years, it has become possible to obtain, as a captured moving image (a moving image obtained by imaging), a moving image (an HDR combine moving image) in which each frame is constituted by an HDR combine image, by performing HDR combination in real time during imaging. FIG. 16 shows how an HDR combine moving image is generated. In the example shown in FIG. 16, first frames and second frames are alternately obtained by imaging, and every time a first frame and a second frame are obtained, the obtained first frame and second frame are combined so as to generate an HDR combine image. Accordingly, an HDR combine moving image is generated.
In a case of generating an HDR moving image, changing weights of a first frame and a second frame in HDR combination enables the degree of motion blur of a subject (a perceived focus of the subject) to be changed in addition to changing a dynamic range. FIG. 17A shows an example of a first frame obtained by imaging of a black circular object moving in a horizontal direction. In the first frame, due to a long exposure time, while gradation information of dark portions can be retained in detail, motion blur is prominent. FIG. 17B shows an example of a second frame obtained by imaging of the black circular object moving in the horizontal direction. In the second frame, due to a short exposure time, gradation information of bright portions can be retained in detail and motion blur is not prominent. FIGS. 17C and 17D show examples of HDR combine images obtained by combining the first frame shown in FIG. 17A and the second frame shown in FIG. 17B. FIG. 17C shows an example of a case where a weight of the first frame is large and a weight of the second frame is small. In this case, as shown in FIG. 17C, an HDR image that retains motion blur is generated. FIG. 17D shows an example of a case where the weight of the first frame is small and the weight of the second frame is large. In this case, as shown in FIG. 17D, an HDR image with smaller motion blur than in FIG. 17C is generated. Generally, in a case of reproducing a moving image at a normal reproduction speed, an image that retains motion blur to a certain degree (such as the image shown in FIG. 17C) is desirably used in order to express natural motion. On the other hand, in a case of reproducing a moving image at a slower reproduction speed than a normal reproduction speed (slow reproduction and pause reproduction), an image with a sharper perceived focus (such as the image shown in FIG. 17D) is desirably used.
In recent years, sizes of imaging sensors in video cameras have been increasing and more and more video cameras enable a wide variety of lenses designed for single-lens reflex cameras to be mounted therein. Against this backdrop, a work flow involving acquiring a high resolution moving image by imaging and extracting a single frame of the moving image as a still image is attracting attention. Hereinafter, such a work flow will be referred to as a “motion capture work flow”. In a motion capture work flow, for example, a high resolution moving image for a commercial or a PV (promotional video) is acquired by imaging and a single frame of the high resolution moving image is extracted as a still image to be used in a magazine or for an advertisement.
FIG. 18 is a diagram showing an example of a configuration of an image display system used in a motion capture work flow. The image display system shown in FIG. includes an imaging apparatus 1100, a reproducing apparatus 1200, a developing apparatus 1300, and an image display apparatus 1400. The imaging apparatus 1100 generates moving image data by imaging and outputs the generated moving image data. As moving image data, the imaging apparatus 1100 outputs RAW moving image data in which image data of each frame is RAW image data. The reproducing apparatus 1200 stores the RAW moving image data output from the imaging apparatus 1100. In addition, the reproducing apparatus 1200 outputs the stored RAW moving image data. The developing apparatus 1300 performs a developing process on the RAW moving image data output from the reproducing apparatus 1200 and outputs the moving image data after the developing process. The image display apparatus 1400 displays a moving image based on the moving image data output from the developing apparatus 1300.
In the image display system shown in FIG. 18, first, RAW moving image data obtained by the imaging apparatus 1100 is accumulated in the reproducing apparatus 1200. Subsequently, the RAW moving image data accumulated in the reproducing apparatus 1200 is developed by the developing apparatus 1300 and a result of the development is displayed by the image display apparatus 1400. A user (for example, a cameraman) checks a captured moving image, selects a still image, or takes other relevant actions by viewing the display of the image display apparatus 1400.
FIG. 19 is a flow chart showing an example of a motion capture work flow. Work performed from a state where normal reproduction of a captured moving image is being conducted will now be described. Normal reproduction refers to a reproduction method in which a moving image is reproduced at a normal reproduction speed.
First, the user checks the display of the image display apparatus 1400 and searches for a group of images (a scene) which may be usable as a still image (S1001; checking work). In addition, the user performs a user operation (a pause operation) for pause reproduction from a frame near the searched group of images and repetitively performs a user operation (a frame-by-frame operation) for frame-by-frame reproduction (S1002). Accordingly, a frame near the searched group of images is displayed and, each time a frame-by-frame operation is performed, the display is switched to a next frame. Next, the user checks the display of the image display apparatus 1400 while performing frame-by-frame operations and selects a frame to be used as a still image (S1003; selection work). Generally, a frame is selected in consideration of a presence or an absence of a perceived focus in addition to a composition of an image. For example, a frame retaining a necessary and sufficient perceived focus for use in a magazine or an advertisement is selected. The user then performs a user operation (an extraction operation) for extracting RAW image data of the selected frame from the reproducing apparatus 1200 as still image data (S1004). Next, the user determines whether or not the motion capture work flow is to be ended (S1005). In a case of ending the motion capture work flow, the present flowchart is ended, but in a case of not ending the motion capture work flow, work is returned to S1002 and a frame-by-frame operation is performed again. Work may be returned to S1001 in a case where the motion capture work flow is not ended. Moreover, RAW image data is extremely suitable for editing images. By extracting RAW image data as still image data, editing work such as retouching in accordance with applications such as magazines and advertisements can be performed in a preferable manner.
In a case of generating an HDR combine moving image in the motion capture work flow, in the checking work in S1001, an HDR combine moving image is favorably generated by using weights suitable for reproducing moving images at a normal reproduction speed. However, the use of such weights often results in displays with an insufficient perceived focus being performed in the selection work in S1003. It is assumed that, in a case where such a display is performed, weights are adjusted by the user.
In a motion capture work flow, selecting a preferable image from an enormous number of images (frames) requires an extremely long period of time and places a huge burden on the user. An added requirement of adjustment of weights in the selection work further increases work time or the burden on the user.
Conventional art related to HDR combination is disclosed in, for example, Japanese Patent Application Laid-open No. 2012-084983 and Japanese Patent Application Laid-open No. 2012-175277. With the technique disclosed in Japanese Patent Application Laid-open No. 2012-084983, weights of an image created by long-time exposure and an image created by short-time exposure are adjusted based on a presence or an absence of motion of a subject in an HDR combine moving image. With the technique disclosed in Japanese Patent Application Laid-open No. 2012-175277, the number of images used in HDR combination and exposure conditions of the images used in the HDR combination are adjusted in accordance with a user operation.
However, with the techniques disclosed in Japanese Patent Application Laid-open No. 2012-084983 and Japanese Patent Application Laid-open No. 2012-175277, a weight is uniquely determined with respect to one frame of an HDR combine moving image. In a case where a weight is uniquely determined with respect to one frame, an unnatural display with a small motion blur may be performed during the checking work in S1001 and a display with an insufficient perceived focus may be performed during the selection work in S1003. Therefore, as a weight with respect to a same frame, different weights are favorably used during checking work and during selection work. Such processing cannot be realized using the techniques disclosed in Japanese Patent Application Laid-open No. 2012-084983 and Japanese Patent Application Laid-open No. 2012-175277 and, as a result, a user operation to adjust weights is required.